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EXECUTIVE  SUMMARY 


OBJECTIVE 

The  operator  of  a  teleoperated  land  vehicle  must  have  a  sufficient  understanding  of  the 
remote  vehicle  attitude  to  successfully  operate  it  in  an  unstructured  outdoor  environment.  The 
goal  of  this  effort  was  to  develop  a  human-machine  interface  that  provides  sufficient  remote 
vehicle  attitude  information  while  minimally  impacting  operator  workload. 

APPROACH 

The  approach  began  with  a  study  of  the  human  orientation  system,  specifically  sensor  stimu¬ 
lation  and  anatomic  sensor  data  processing.  Potential  methods  for  presenting  vehicle  attitude 
information  to  the  operator  that  stimulate  the  human  orientation  system  were  identified  and 
explored. 

The  method  of  gravity-referencing  the  remote  sensor  package  was  implemented.  This 
method  involved  referencing  the  remote  visual  sensors  to  the  gravitational  field  of  the  earth.  The 
hood  of  the  remote  vehicle  appeared  in  the  video  image  transmitted  to  the  control  station  and 
provided  the  operator  with  vehicle  pitch  and  roll  information.  The  effectiveness  of  this  method 
versus  vehicle-referencing  the  sensors  was  experimentally  investigated. 

RESULTS  AND  RECOMMENDATIONS 

Statistically  significant  improvement  (with  a  99.9  percent  confidence  level)  was  observed  in 
the  operator’s  understanding  of  remote  vehicle  attitude,  both  pitch  and  roll,  when  the  sensor 
package  was  gravity-referenced  compared  with  when  it  was  vehicle-referenced. 

The  improvement  in  the  operator’s  understanding  of  the  vehicle  roll  angle  with  gravity- 
referenced  sensors  was  greater  than  that  for  the  pitch  angle  and  was  operationally  significant.  To 
enhance  the  effectiveness  of  the  gravity-referencing  method  for  pitch  angle  estimation,  a  simple 
graphic  indicator  could  be  overlaid  on  the  edge  of  the  video  display,  providing  angular  posi¬ 
tion  meaning  to  the  linear  movement  of  the  vehicle  hood  in  the  video  image.  Although  the 
graphical  indicator  was  not  formally  tested,  informal  investigation  indicated  that  the 
gravity-referencing  method  coupled  with  the  simple  graphical  indicator  would  provide  an  opera¬ 
tionally  significant  improvement  in  pitch  angle  understanding. 

Experimental  results  showed  a  significantly  higher  level  of  operator  confidence  in  vehicle 
attitude  awareness  with  the  gravity-referencing  method.  A  minimal  increase  in  operator  work¬ 
load  was  also  documented  for  the  method  of  gravity-referencing. 
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1.  INTRODUCTION 


Teleoperation  is  the  manipulation  of  a  remote  system  from  a  controller  site.  The  motivation 
behind  it  is  simple — to  remove  the  human  worker  from  an  inconvenient,  dangerous,  or  unreach¬ 
able  work  site  while  retaining  control  and  decision-making  capabilities  over  a  situation. 

1.1  TELEOPERATED  LAND  VEHICLES 

Much  of  the  modern  battlefield,  with  smart  weapons  and  nuclear-biological-chemical 
(NBC)  munitions,  is  becoming  too  hostile  for  humans  to  survive  and  contribute  to  the  outcome 
of  the  conflict  [Shaker  and  Wise,  1988].  The  Department  of  Defense  (DoD)  has  realized  the  need 
to  replace  soldiers  engaged  in  missions  or  located  in  environments  with  little  survivability  and  is 
investing  in  programs  to  develop  robotic  systems  for  high-risk  or  hostile  battlefield  applications 
[Aviles  et  al.,  1990;  DoD,  1991]. 

Robotic  vehicles  can  be  autonomous  with  remote  vehicle  intelligence,  semi-autonomous  with 
intermittent  teleoperated  control,  or  teleoperated  with  little  remote  vehicle  intelligence.  Both 
teleoperated  and  autonomous  systems  have  strong  points  and  limitations  that  make  them  suitable 
for  different  battlefield  missions. 

Teleoperation  allows  the  military  command  to  have  complete  and  continuous  control  over 
vehicle  actions  and  movement.  Thus,  the  system  is  robust  and  adaptive  in  an  unstructured 
environment  or  unforeseen  scenario.  Responses  of  force  are  initiated  and  controlled  by  a  human 
operator;  therefore,  there  need  not  be  any  concern  of  a  machine  inappropriately  directing  force. 
The  communications  link  between  the  vehicle  and  operator  is  vulnerable  to  breakage  or  jam¬ 
ming;  if  the  link  becomes  nonfunctional,  the  system  is  useless.  Teleoperated  vehicles  would  be 
suitable  for  reconnaissance,  surveillance,  and  target  acquisition  missions  immediately  ahead  of 
troops.  With  current  communication  link  configurations,  the  teleoperated  system  would  not  be 
effective  for  deep  penetration  behind  enemy  lines.  Autonomous  systems  are  better  suited  for  that 
type  of  mission.  Both  autonomous  and  teleoperated  vehicles  could  be  used  to  draw  fire  away 
from  troops.  [Shaker  and  Wise,  1988] 

Although  the  development  of  teleoperated  land  vehicles  has  been  focused  on  military 
applications,  unmanned  land  vehicles  are  beginning  to,  or  may  soon,  find  a  place  in  space 
exploration,  drug  enforcement,  border  patrol,  and  security  surveillance  activities  [Courtright, 
1991]. 

1.2  REMOTE  VEHICLE  ATTITUDE  AWARENESS 

One  of  the  critical  technology  areas  in  teleoperated  land  vehicle  development  is  the  human- 
machine  interface  [DoD,  1991].  A  key  issue  in  the  human— machine  interface  design  is  the 
operator’s  insufficient  understanding  of  the  remote  vehicle  attitude  [Aviles  et  al.,  1990].  This 
lack  of  awareness  is  a  roadblock  for  the  safe  and  effective  operation  of  teleoperated  ground 
vehicles  over  rough  terrain  and  in  off-road  situations.  Teleoperated  vehicle  roll-over  accidents, 
attributable  to  insufficient  vehicle  attitude  cues  being  presented  to  the  operator,  have  been 
documented  by  Sandia  National  Laboratories  [McGovern,  1990].  Also,  the  Naval  Command, 
Control,  and  Ocean  Surveillance  Center’s  Research,  Development,  Test,  and  Evaluation  Division 
(NCCOSC  RDT&E)  has  witnessed  and  documented  near  roll-over  incidents  due  to  insufficient 
vehicle  attitude  awareness  [Aviles  et  al.,  1990], 
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A  teleoperated  vehicle  operator  extracts  remote  vehicle  attitude  cues  from  a  video  display;  in 
addition,  some  teleoperator  control  consoles  are  equipped  with  a  vehicle  pitch  and  roll  indicator. 
This  configuration  does  not  supply  sufficient  cues  for  the  operator  to  understand  and  react  to  the 
continuously  changing  attitude  of  the  remote  vehicle  as  it  traverses  rough  terrain  [Aviles  et  al., 
1990]. 

1.3  OBJECTIVE 

The  goal  of  this  effort  was  to  develop  a  human-machine  interface  that  provides  remote 
vehicle  attitude  information,  sufficient  for  the  successful  teleoperation  of  the  system,  while 
minimally  impacting  operator  workload. 

This  report  begins  with  a  review  of  the  human  orientation  system,  specifically  sensor  stimu¬ 
lation  and  anatomic  sensor  data  processing.  Much  of  the  literature  in  this  area  is  from  the 
aviation  medical  community.  Potential  methods  for  presenting  vehicle  attitude  information  to  the 
operator  that  stimulate  the  human  orientation  system  are  identified  and  explored. 

Next,  the  selected  method  is  discussed.  This  method  involved  referencing  the  remote  visual 
sensors  to  the  gravitational  field  of  the  earth.  The  hood  of  the  remote  vehicle  appeared  in  the 
video  image  transmitted  to  the  control  station  and  provided  the  operator  with  vehicle  pitch  and 
roll  information.  A  two-degree-of-freedom  platform  for  referencing  the  cameras  was  designed 
and  prototyped.  The  system  responded  only  to  changes  in  terrain  slope  and  ignored  transient 
bumps,  so  that  information  about  terrain  roughness  was  not  lost. 

The  prototype  hardware  was  integrated  on  a  teleoperated  ground  vehicle  test  bed.  The 
effectiveness  of  gravity-referencing  the  remote  sensors  was  experimentally  investigated.  Results 
are  presented  and  reviewed. 

Finally,  suggestions  are  offered  for  the  implementation  of  the  gravity-referencing  method  on 
teleoperated  land  vehicle  systems. 
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2.  PRELIMINARY  INVESTIGATIONS 


The  first  step  was  to  study  the  human  sensory  system,  which  provides  orientation  informa¬ 
tion.  When  an  operator  is  directly  driving  a  vehicle,  the  human  body  collects  and  processes 
-  orientation  data.  This  information  enables  the  operator  to  understand  the  orientation  of  the 
vehicle.  When  the  operator  is  teleoperating  a  vehicle,  orientation  cues  must  be  recreated  or 
artificial  cues  generated  at  the  control  console  for  the  operator  to  sufficiently  understand  the 
attitude  of  the  remote  vehicle. 

2.1  HUMAN  ORIENTATION  SYSTEM 

The  collection  and,  in  some  cases,  processing  of  orientation  information  are  carried  out  by 
the  visual,  vestibular  and  nonvestibular  proprioceptors,  mechanoreceptors,  and  auditory  subsys¬ 
tems. 

Vision  has  been  identified  as  the  most  important  subsystem  in  spatial  orientation  awareness 
[Gillingham  and  Wolfe,  1986].  The  anatomic  sensors  associated  with  vision  are  cones  and  rods. 
Cones  are  responsible  for  sharp  visual  acuity  and  color  vision;  rods  provide  less  visual  discrimi¬ 
nation  but  are  more  sensitive  to  low  light  levels.  The  visual  subsystem  is  composed  of  two 
separate  and  independent  systems,  with  distinct  principal  functions. 

Peripheral,  or  ambient,  vision  is  predominately  responsible  for  spatial  orientation.  Both  rods 
and  cones  provide  the  sensor  basis  for  peripheral  vision. 

Foveal,  or  focal,  vision  is  involved  with  object  recognition.  Although  not  primarily  responsi¬ 
ble  for  spatial  orientation,  foveal  vision  is  used,  at  times,  to  gather  cues  about  orientation,  e.g.,  an 
airplane  pilot  viewing  pitch,  roll,  and  yaw  instruments.  In  this  employment  of  foveal  vision,  a 
learned  complex  cognitive  process  is  required  to  translate  the  collected  data  into  usable  orienta¬ 
tion  information.  The  sensor  foundation  for  focal  vision  is  the  fovea  centralis,  the  highest  density 
area  of  cones  located  near  the  posterior  pole  of  the  eye  [Gillingham  and  Wolfe,  op.  cit.]. 

The  vestibular  system  detects  accelerations  of  the  body  and  processes  the  data  into  orienta¬ 
tion  information.  This  information  is  then  employed  for  vision  stabilization  and  for  the  execution 
of  voluntary  and  reflexive  motion.  In  the  absence  of  vision,  vestibular  cues  can  usually  provide  a 
reasonable  picture  of  one’s  position  and  orientation  [Gillingham  and  Wolfe,  op.  cit.]. 

Nonvestibular  proprioceptors  include  the  muscle,  tendon,  and  joint  sensors.  Like  the 
vestibular  system,  they  are  sensitive  to  accelerations.  The  information  they  collect  is  integrated 
with  the  vestibular  proprioceptor  information  [Gillingham  and  Wolfe,  op.  cit.]. 

Mechanoreceptors  are  pressure-sensitive  sensors  in  the  skin.  The  information  they  gather  is 
also  combined  with  that  of  the  vestibular  system  [Gillingham  and  Wolfe,  op.  cit.]. 

The  binaural  auditory  subsystem  utilizes  magnitude,  phase,  and  arrival  time  differences  to 
localize  a  sound  source.  Also,  a  rotating  sound  from  a  stationary  source  can  be  detected  and 
translated  into  a  rotating  motion  of  the  body.  In  addition,  one  can  interpret  specific  sounds  to 
have  meaning,  e.g.,  an  audio  warning  alarm  on  an  instrument  panel,  but  this  requires  a  learned 
complex  cognitive  process. 

2.2  ORIENTATION  CUES  FOR  TELEOPERATION 

After  obtaining  a  basic  understanding  of  the  human  orientation  system,  the  second  step  in  the 
effort  was  to  identify  methods  of  presenting  remote  vehicle  attitude  data  in  such  a  manner  as  to 
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stimulate  the  operator’s  orientation  sensors.  For  the  purpose  of  this  effort,  the  human  subsystems 
were  grouped  into  three  categories:  visual,  proprioceptors  (vestibular  and  non-vestibular)  and 
mechanoreceptors,  and  auditory.  Methods  of  presenting  orientation  cues  were  identified,  then 
classified  as  a  stimulant  of  one  of  the  three  categories  of  human  orientation  subsystems.  Potential 
methods  are  listed  in  table  2.1. 


Table  2-1.  Potential  methods  for  vehicle  attitude  feedback. 


Vision 

Proprioceptors, 

Mechanoreceptors 

Audio 

Stereo  display 

Motion  seat 

Binaural  display 

Color  display 

Wide  field-of-view  display 

Force  reflective  vehicle 
controls 

Warning  and  alarm 
sounds 

Gravity-referenced  vision 
sensors 

Numeric  or  graphic  attitude 
indicator 

Video  gravity-referencing  was  selected  as  the  initial  method  to  be  further  investigated  to 
determine  its  effectiveness  in  providing  remote  vehicle  attitude  information  to  the  operator.  This 
selection  does  not  dismiss  the  other  methods;  the  research  had  to  begin  by  prototyping  and 
testing  one  “good  candidate”  method.  This  method  stimulates  the  operator’s  peripheral  vision 
subsystem,  requires  no  changes  to  a  control  console  equipped  with  a  video  display,  and  requires 
no  additional  communication  channels  between  the  remote  vehicle  and  the  control  site. 
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3.  PROTOTYPE  PLATFORM  DESIGN 


A  prototype  gravity-referencing  platform  was  developed  and  integrated  on  a  teleoperator  test 
bed  at  NCCOSC  RDT&E,  Kaneohe  Detachment.  The  test  vehicle  was  a  High  Mobility  Multi- 
Purpose  Wheeled  Vehicle  (HMMWV),  equipped  with  color,  stereo  vision,  and  binaural  audio 
sensors.  The  sensors  were  mounted  on  the  referencing  platform.  The  platform  was  located  in  the 
area  between  the  traditional  driver’s  and  passenger’s  seats.  Sensors  were  mounted  at  an  elevation 
equal  to  the  average  height  of  the  eyes  and  ears  of  a  6-foot-tall  human  in  the  driving  position 
[Diffrient,  Tilley,  and  Bardagjy,  1974]. 

The  hood  of  the  remote  vehicle  appeared  in  the  video  image  and  served  as  an  artificial 
horizon  indicator,  providing  the  operator  with  vehicle  pitch  and  roll  information.  The  cameras 
were  positioned  so  that  the  hood  of  the  vehicle  filled  the  lower  one-third  of  the  video  frame 
when  the  vehicle  was  on  level  terrain.  The  platform  did  not  stabilize  the  remote  sensors;  it 
responded  to  changes  in  terrain  slope  but  not  to  transient  bumps.  A  stabilization  scheme  would 
forfeit  information  on  road  roughness. 

3.1  REQUIREMENTS 

A  two-degree-of-freedom-actuated  platform  was  designed  to  gravity-reference  the  remote 
vision  sensors.  The  design  was  subject  to  the  performance,  operational,  and  physical  require¬ 
ments  described  in  the  following  three  sections. 

3.1.1  Performance 

The  platform  was  implemented  on  an  HMMWV.  The  vehicle  was  capable  of  transiting 
60-percent  pitch  axis  grades  (~30  degrees)  and  40-percent  roll  axis  grades  (~20  degrees). 

To  effectively  gravity-reference  the  remote  sensor  package,  the  platform  must  be  capable  of 
the  same  range  of  motion  as  the  vehicle,  ±30  degrees  (0.52  radian)  pitch  and  ±20  degrees 
(0.35  radian)  roll. 

The  aggregate  system  accuracy  specification  for  angular  positioning  was  determined  by 
considering  the  accuracy  of  candidate  position  sensors  and  the  accuracy  required  for  testing 
purposes.  The  figure  arrived  at  was  ±1.5  degrees  (0.026  radian),  which  is  both  reasonable  to 
implement  in  hardware  and  sufficient  for  testing  vehicle  attitude  awareness. 

The  slew  rate  for  the  prototype  platform  was  determined  by  considering  the  test  scenario  and 
terrain  used  for  data  collection.  An  HMMWV  equipped  with  a  rate  sensor  was  driven  over 
rough,  off-road  terrain  at  20  to  25  mph.  The  maximum  rate  of  change  of  the  terrain  recorded  was 
60  deg/s.  The  test  data  were  to  be  collected  at  an  average  vehicle  speed  of  7  mph.  Therefore,  a 
platform  slew  rate  of  20  deg/s  (0.35  rad/s)  should  be  sufficient  to  track  the  maximum  rate  of 
change  of  the  representative  terrain. 

The  duty  cycle  for  each  actuator  was  developed  in  conjunction  with  the  motor  selection.  The 
specified  duty  cycle  accommodated  the  slew  rate  requirement  for  angle  changes  as  small  as 
1.4  degrees,  i.e.,  it  would  take  0.07  second  for  the  actuator  to  move  1.4  degrees.  The  acceleration 
required  was  1146  deg/s2  (20.0  rad/s2). 

The  sensors  used  to  detect  angular  position  of  the  vehicle  must  have  a  linear  output  over  the 
range  of  motion  required  of  the  platform.  Also,  the  update  rate  of  the  sensor  must  be  at  least 
twice  the  frequency  of  the  terrain  (  >  2*20  hertz  =  40  hertz). 
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The  prototype  platform  must  operate  within  performance  guidelines  with  the  vision  and 
auditory  sensor  package  load. 

3.1.2  Operational 

Test  data  were  to  be  collected  at  Bellows  Air  Force  Station  in  Waimanalo,  Hawaii,  over  dirt 
roads  and  HMMWV-blazed  terrain  (the  details  are  described  in  Section  4.2,  Data  Collection  and 
Processing).  The  platform  and  the  data  collection  instrumentation  would  be  required  to  operate 
continuously  for  up  to  6  hours,  with  a  maximum  air  temperature  of  95  °F  and  a  vehicle  surface 
temperature  (for  hardware  mounts)  of  110°F.  In  addition,  the  hardware  would  be  subjected  to  a 
rough  ride  and  very  dusty  conditions. 

The  platform  had  to  be  easy  to  calibrate  and  manually  control  in  the  field.  The  alternation 
between  gravity-  and  vehicle-referencing  techniques  needed  to  be  simple  and  quick. 

3.1.3  Physical 

The  platform  developed  was  a  prototype;  it  was  installed  on  an  existing  HMMWV  vehicle 
that  was  used  for  teleoperator  development.  As  a  result,  the  platform  would  be  subjected  to  some 
physical  size  requirements  so  that  it  could  be  mounted  in  the  available  space  and  still  operate 
effectively  for  the  assigned  task. 

3.2  HARDWARE 

The  physical  size  constraint  for  the  platform  was  the  major  factor  that  led  to  the  design 
illustrated  in  figure  3-1.  The  actuators,  located  directly  under  the  rotating  plate,  used  a  linear 
“pushing”  action  to  achieve  the  required  angular  range  of  motion. 


Figure  3-1.  Two-degree-of-freedom  platform,  front  view. 
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The  platform  was  constructed  from  aluminum,  except  for  the  push-rods,  cams,  universal 
joints,  and  primary  support  post;  these  were  constructed  from  steel,  for  strength.  Mechanical 
springs  were  positioned  on  each  axis  to  reduce  the  lateral  motion  introduced  into  the  system  by 
the  push  rod  and  cam  connection.  Limit  switches  were  mounted  in  the  path  of  the  rotating  cam, 
on  either  side  of  the  actuator  base.  The  switches  prevented  rotational  movement  in  excess  of  the 
platform’s  physical  limits. 

Each  axis  was  actuated  by  a  dc  motor  coupled  with  a  harmonic  gear  head.  The  motor  and 
gear  head  combination  was  selected  to  meet  the  load  and  performance  requirements.  A  two-de- 
gree-of-freedom  inclinometer  was  used  for  pitch  and  roll  detection.  The  unit  integrated  the 
velocity  data  from  a  solid-state  angular  rate  sensor  for  fast  angular  position  updates.  In  addition, 
it  had  a  pendulum  sensor  that  was  used  to  recalibrate  the  integrated  rate  sensor  data  every 
60  seconds.  This  coupled  design  afforded  fast  and  accurate  angular  position  information.  The 
update  rate  of  the  sensor  was  55  hertz.  The  sensor  had  a  linear  angular  position  output  range  of 
±  30  degrees,  with  an  accuracy  rating  of  less  than  ±  0.9  degree.  The  rate  sensor  component 
saturated  at  a  frequency  of  100  deg/s. 

3.3  CONTROL 

A  control  algorithm  was  developed  to  calculate  the  torque  necessary  to  move  each  joint  from 
its  present  position  to  a  new  position  along  a  specified  trajectory.  The  designed  closed-loop 
proportional-differential  cascade  control  algorithm  satisfied  the  performance  requirements  for 
the  platform. 

The  control  strategy  was  implemented  in  analog  electronics.  The  actuators  and  controller 
operated  on  24  volts  supplied  from  two  12-volt  lead-acid  batteries  wired  in  series.  The  power 
requirement  was  7  amps  at  24  volts. 

The  control  electronics  were  designed  to  accept  voltage  inputs  from  two  angular-position 
sensors  or  from  two  hand-turned  potentiometers.  Each  axis  had  a  select  switch  for  the  input 
source.  This  configuration  provided  for  efficient  switching  between  gravity-  and  vehicle- 
referencing  methods  and  for  quick  field  calibration. 
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4.  EFFECTIVENESS  OF  GRAVITY-REFERENCING 


A  full-factorial  experiment  was  conducted  to  compare  the  effectiveness  of  a  teleoperator 
system  with  gravity-referenced  sensors  to  the  same  teleoperator  system  with  vehicle-referenced 
sensors  (i.e.,  sensors  fixed  relative  to  the  vehicle  and  subject  to  the  same  motion).  The  test 
subject’s  error  in  estimating  the  pitch  and  roll  angles  was  the  parametric  used  to  measure  the 
effectiveness  of  the  methods. 

4.1  EXPERIMENTAL  DESIGN 

The  experiment  was  designed  to  emulate  a  typical  remote  vehicle  driving  scenario  in  an 
unstructured  environment.  This  scenario  was  to  be  a  dynamic  situation  requiring  an  on-the-fly 
operator  response:  The  vehicle  is  being  teleoperated  over  rough,  unfamiliar  terrain.  The  operator 
must  continually  be  aware  of  the  vehicle  attitude  to  determine  if  continued  operation  is  safe, 
whether  to  reduce  or  increase  speed,  and  if  an  alternative  route  must  be  pursued. 

For  cost  and  safety  reasons,  the  experiment  was  conducted  in  the  laboratory  using  field- 
collected  data. 

4.2  DATA  COLLECTION  AND  PROCESSING 

The  HMMWV  was  taken  to  Bellows  Air  Force  Station  and  driven  over  rough  dirt  roads  and 
HMMWV-blazed  terrain.  Stereo,  color  video,  binaural  audio,  and  vehicle  pitch  and  roll  data 
were  collected.  Driving  courses  were  traversed  twice  at  a  speed  of  7  ±  3  mph,  alternating 
between  gravity-referencing  the  sensors  and  vehicle-referencing  the  sensors.  Figure  4.1  shows 
the  data-collection  instrumentation. 

Stereo  video  was  recorded  on  Recorder  1  using  a  field  sequential  mixer.  Monocular  video 
was  recorded  directly  on  Recorder  2.  The  binaural  audio  was  recorded  on  the  audio  tracks  of 
Recorder  1.  The  pitch  and  roll  data  were  measured  with  the  same  model  of  inclinometer  used  on 
the  two-degree-of-freedom  platform  described  in  Section  3.2,  Hardware.  The  voltage  signals 
were  converted  to  corresponding  frequencies,  then  recorded  on  the  two  audio  tracks  of 
Recorder  2.  A  clapboard  was  used  at  the  begining  of  each  clip  so  that  the  video  signals  on 
Recorder  1  and  Recorder  2  could  be  synchronized  during  the  editing  process. 

The  data  were  edited  into  clips  and  transferred  to  two  optical  disks,  one  for  gravity-  and  the 
other  for  vehicle-referenced  data.  The  optical  disk  medium  was  chosen  to  enable  the  random 
selection  and  playback  of  the  clips  during  testing. 

4.3  EXPERIMENTAL  PROCEDURE 

Each  test  subject  received  an  introductory  brief,  both  verbal  and  written;  the  written  explana¬ 
tion  is  included  in  the  Appendix.  Also,  every  subject  signed  a  voluntary  consent  form  and 
completed  a  pretest  questionnaire  (details  are  presented  in  Section  4.4,  Test  Subjects).  After  each 
test  session,  the  subjects  were  asked  to  complete  a  posttest  questionnaire.  Both  questionnaires 
are  included  in  the  Appendix. 

Each  subject  participated  in  two  practice  sessions  followed  by  two  20-minute  test  ses¬ 
sions.  Each  of  the  two  test  sessions  (as  well  as  the  practice  sessions)  featured  a  different 
method — gravity-  or  vehicle-referencing — of  presenting  remote  vehicle  attitude  information 
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Figure  4-1.  Field  data-collection  instrumentation. 


via  video  and  audio  displays.  Each  test  session  consisted  of  14  driving  scenarios,  45  seconds  in 
length.  At  three  random  points  during  each  45-second  scenario,  the  video  and  audio  stopped  and 
the  test  subject  made  an  estimate  of  the  HMMWV  pitch  and  roll  angles.  The  test  subject 
expressed  estimates  by  positioning  a  gimballed  model  vehicle  (see  figure  4-2)  in  the  same  spatial 
orientation  as  the  observed  HMMWV. 

Testing  was  conducted  in  the  laboratory,  as  shown  in  figure  4-3.  The  test  subject  was  seated 
at  a  distance  behind  a  21-inch  color  monitor  to  match  the  39-degree  horizontal  field-of-view  at 
which  the  video  was  collected.  The  subject  wore  flicker  glasses  (for  field  sequential  stereo 
viewing)  and  stereo  headphones  (for  binaural  hearing).  The  subject  positioned  the  model  vehicle 
with  one  or  both  hands,  whichever  was  easier  and  more  comfortable.  When  the  vehicle  was 
positioned,  the  subject  depressed  a  hand-held  switch,  and  a  pitch  and  roll  reading  was  taken  of 
the  orientation  of  the  model  vehicle. 

The  randomized  experimental  design  was  implemented  with  control  software  developed  on  a 
Macintosh  Ilfx  using  National  Instruments  Labview  II  programming  environment.  The  gravity- 
referenced  driving  scenario  clips  were  stored  on  one  optical  disk  and  the  vehicle-referenced  clips 
were  stored  on  another.  The  clips  were  called  up  and  presented  to  the  test  subject  in  a  random, 
but  continuous  fashion.  The  gravity-referenced  clips  were  shown  first  to  alternate  subjects;  the 
other  subjects  were  shown  the  vehicle-referenced  clips  first.  Estimated  pitch  and  roll  data  were 
collected  via  an  analog-to-digital  computer  board  and  converted  from  a  voltage  to  a  correspond¬ 
ing  angle  measurement.  Actual  pitch  and  roll  data  were  extracted  from  the  audio  tracks,  con¬ 
verted  from  a  frequency  to  a  corresponding  voltage,  and  were  stored  in  an  input  data  file.  Test 
instrumentation  is  diagramed  in  figure  4-4. 
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Figure  4-2.  Gimballed  model  vehicle  used  to  input  estimates  of  pitch 
and  roll  angles. 


Figure  4-3.  Test  operator’s  station. 
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Figure  4-3.  Test  operator’s  station. 


Figure  4-4.  Experiment  instrumentation. 


4.4  TEST  SUBJECTS 

The  12  test  subjects  were  recruited,  on  a  voluntary  basis,  from  NCCOSC  RDT&E,  Kaneohe 
Detachment,  and  the  University  of  Hawaii  at  Manoa.  None  of  the  test  subjects  were  familiar  with 
the  effort.  Since  orientation  feedback  stimuli  for  the  experiment  were  to  be  presented  in  stereo 
video  and  binaural  audio  formats,  the  subject  candidates  were  screened  for  uncorrected  vision 
and  hearing  problems  and  also  for  difficulties  with  stereoscopic  vision. 

A  pretest  questionnaire  was  administered  to  characterize  the  subject  pool.  The  information  is 
introduced  in  table  4-1. 

4.5  EXPERIMENTAL  ANALYSIS 

Each  test  subject  generated  42  angle  estimation  data  points  for  each  of  the  two  test  sessions, 
for  a  total  sample  size  of  1008  data  points.  Additional  information  collected  for  each  data  point 
included  subject  identification  number,  test  session  number,  referencing  method,  and,  of  course, 
actual  angle  data. 

As  stated  earlier,  the  test  subject’s  error  in  estimating  the  pitch  and  roll  angles  was  the 
parametric  used  to  measure  the  effectiveness  of  the  methods.  Methods  of  statistical  inference 
were  employed  to  show  that  the  variance  in  the  angle-estimation  errors  was  attributable  to  the 
referencing  method  and  not  just  random,  or  chance,  variation. 
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Table  4-1.  Test  operator  characteristics. 


Subject 
ID  No. 

Occupation 

No.  Years  as 
Licensed 
Driver 

Suscep¬ 
tibility  to  Motion 
Sickness 

HMMWV 

Experience 

Off-Road  Driving 
Experience 
as  Driver 

1 

Student 

8 

Moderate 

Observer 

None 

2 

Administrative 

Assistant 

8 

Moderate 

Observer 

None 

3 

Engineer 

18 

Moderate 

Driver 

Minimal 

4 

Engineer 

30 

Moderate 

Driver 

Minimal 

5 

Engineer 

14 

Minimal 

Driver 

Minimal 

6 

Engineer 

29 

Minimal 

Observer 

Substantial 

7 

Engineer 

12 

Minimal 

Observer 

Minimal 

8 

Administrative 

Assistant 

21 

High 

Observer 

Minimal 

9 

Financial 

Specialist 

16 

Minimal 

Observer 

None 

10 

Computer 

Scientist 

14 

Minimal 

Observer 

Moderate 

11 

Mechanical 

Technician 

40 

Nonexistent 

Driver 

None 

12 

Student 

19 

Minimal 

Observer 

None 

The  null  hypothesis  (Ho)  developed  for  this  study  is  that  there  are  no  differences  between  the 
values  of  angle  estimation  error  ( the  dependent  variable)  that  can  be  explained  by  differences  in 
sensor  referencing  techniques,  test  operators,  or  test  session  order  ( the  independent  variables,  or 
treatments). 

An  analysis  of  variance  (ANOVA)  was  done  on  the  collected  data.  ANOVA  is  a  technique 
for  identifying  the  sources  of  variation  that  contribute  to  the  total  variation  of  a  data  set.  It  is  a 
tool  for  testing  the  null  hypothesis  against  a  collection  of  alternative  hypotheses. 

When  significant  differences  are  observed  in  the  values  of  the  dependent  variable  that  are 
attributable  to  independent  variables,  the  null  hypothesis  is  rejected.  If  no  significant  differences 
are  discovered,  the  null  hypothesis  is  not  proven  true,  but  is  tentatively  accepted. 

The  ANOVA  test  partitions  the  total  sample  variance  into  the  “between”  treatment  variance, 
or  treatment  mean  sum  of  squares  (MSTr),  and  the  “within”  treatment  variance,  or  error  mean 
sum  of  squares  (MSE).  The  between  treatment  MSTr  represents  the  variance  due  to  the  effective¬ 
ness  of  the  treatments  and  the  within  MSE  expresses  the  variance  due  to  chance.  The  ratio  of 
MSTr  to  MSE  produces  a  variable  which  follows  the  F-distribution  [Huntsberger,  1967;  Miller 
and  Freund,  1977].  The  ANOVA  uses  the  observed  F-value  to  decide  on  the  rejection  of  the  null 
hypothesis.  The  observed  F-value  is  compared  with  the  theoretical  F-value.  The  probability  that 
an  F-value  as  large  as  the  observed  F-value  could  occur  by  chance  (if  the  null  hypothesis  is  not 
rejected)  is  calculated.  A  typical  cutoff  level  employed  for  reporting  statistically  significant 
differences  (i.e.,  rejecting  Hq )  is  0.05  [Gagnon  et  al.,  1989];  this  means  that  only  5  percent  of  the 
time,  Hq  would  be  incorrectly  rejected. 
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Since  implementing  a  rejection  of  the  null  hypothesis  for  this  experiment  could  mean  a 
hardware  change  to  teleoperated  vehicles,  a  stricter  cutoff  level  of  0.01  was  adopted  for  the 
rejection  of  Hq.  This  means  that  only  1  percent  of  the  time  Hq  would  be  incorrectly  rejected. 

The  primary  null  hypothesis  was  investigated,  and  each  subject’s  vehicle  attitude  awareness 
in  the  critical  ranges  (i.e.,  pitch  and  roll  angles  exceeding  10  degrees)  was  evaluated.  On  the 
posttest  questionnaire,  subjects  were  asked  to  rate  their  level  of  confidence  regarding  their 
estimates  and  to  rate  the  task  workload  of  making  estimates  for  the  two  methods.  These  results 
are  also  reported. 

4.6  EXPERIMENTAL  RESULTS 

Vehicle  pitch  and  roll  estimation  errors  were  investigated  separately.  ANOVAs  were 
conducted  with  roll  and  pitch  estimation  errors  as  the  dependent  variables  and  the  subject 
identification  number,  test  session  number,  and  referencing  method  as  independent  effects. 

4.6.1  Learning  and  Order  Effect 

The  12  test  subjects  were  each  assigned  a  number.  The  odd-numbered  subjects  were  first 
tested  on  the  gravity-referencing  method  and  then  on  the  vehicle-referencing  method.  The  order 
of  testing  was  reversed  for  the  even-numbered  subjects.  This  was  done  to  reduce  any  effects  of 
order  and  learning.  Results  of  the  ANOVAs — table  4.2  for  roll  estimation  and  table  4.3  for  pitch 
estimation — confirmed  that  differences  in  angle  estimation  errors  were  not  attributable  to  test 
session  order. 


Table  4-2.  ANOVA  for  roll  estimation  error. 


Source 

Degree- 

of-Freedom 

Sum  of 
Squares 

Mean  Square 

F- Value 

p- Value 

Reference 

Method 

1 

3512.693 

3512.693 

206.590 

0.0001 

Subject  ID 

11 

971.065 

88.279 

5.192 

0.0001 

Session 

1 

31.609 

31.609 

1.859 

0.1730 

Error 

994 

16,901.220 

17.003 

Dependent:  Roll  Estimation  ’ 

Error 

Table  4-3.  ANOVA  for  pitch  estimation  error. 

Source 

Degree- 

of-Freedom 

Sum  of 
Squares 

Mean  Square 

F- Value 

/7-Value 

Reference 

Method 

1 

503.910 

503.910 

28.013 

0.0001 

Subject  ID 

11 

1622.451 

147.496 

8.200 

Session 

1 

4.204 

4.204 

0.234 

Error 

994 

17,880.216 

17.988 

Dependent:  Pitch  Estimation  Error 
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The  mean  errors  for  the  two  sessions  are  shown  in  tables  4-4  and  4-5,  for  roll  and  pitch, 
respectively. 


Table  4-4.  Session  order  means  for  roll  estimation  error 


Session 

Count 

Mean 

Std.  Dev. 

Std.  Error 

First 

5.064 

4.375 

0.195 

5.419 

4.835 

0.215 

Dependent:  Rol 

l  Estimation  Error  (degrees) 

Table  4-5.  Session  order  means  for  pitch  estimation  error. 


Session 

Count 

Mean 

Std.  Dev. 

Std.  Error 

First 

6.470 

4.401 

0.196 

6.341 

4.518 

0.201 

Dependent:  Pitch  Estimation  Error  (degrees) 


4.6.2  Referencing  Method  Effect 

The  ANOVA  results  displayed  in  tables  4-2  and  4-3  indicate  that  the  null  hypothesis 
(outlined  in  Section  4.5,  Experimental  Analysis)  can  be  rejected  for  the  referencing  method 
effect  and  the  test  subject  effect.  The  variance  attributable  to  test  subject  differences  is  explored 
in  the  Section  4.6.3,  Test  Subject  Effect.  The  variance  due  to  referencing  methods  is  studied 
here. 

4.6.2.1  Roll  Axis.  The  roll  axis  results  are  explored  first.  The  high  F  value  and  th ep  value  in 
table  4-2  indicate  that  H0  can  be  rejected  and  the  probability  that  it  is  being  incorrectly  rejected 
is  only  0.01  percent.  Thus,  the  alternative  hypothesis— that  the  differences  in  the  roll  estimation 
error  is  dependent  upon  the  referencing  method — is  accepted  with  a  confidence  level  of  99.99 
percent.  This  significance  level  is  much  smaller  than  the  critical  cutoff  level  of  0.01  (or  1  per¬ 
cent)  specified  for  the  experiment  in  Section  4.5,  Experimental  Analysis. 

The  mean  vehicle  roll  estimation  errors  are  shown  in  table  4-6  for  gravity-referenced  sensors 
and  vehicle-referenced  sensors. 

Table  4-6.  Reference  method  means  for  roll  estimation  error. 


Reference 

Method 

Count 

Mean 

Std.  Dev. 

Std.  Error 

Vehicle 

504 

7.108 

5.323 

0.237 

Gravity 

504 

3.375 

2.695 

0.120 

Dependent:  Roll  Estimation  Error  (degrees) 
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It  is  clear  that  the  gravity-referencing  method  is  facilitating  more  accurate  roll  estimations 
than  the  vehicle-referencing  method  for  the  entire  range  of  vehicle  roll  angles  (i.e.,  0  to  20 
degrees  left  and  right  side  slopes).  Although  the  operator’s  awareness  of  vehicle  attitude  is 
always  important,  it  becomes  critical  when  the  vehicle  is  traversing  side  slopes  of  significant 
angle. 

Figure  4-5  displays  cell  mean  (average  estimation  error  for  each  actual  angle)  estimation 
errors  plotted  against  the  actual  roll  angle.  The  estimation  errors  associated  with  gravity- 
referencing  fall  within  a  narrow  band  across  the  entire  side  slope  range.  The  errors  are  consistent 
and  do  not  correspond  to  actual  side  slope  angles.  The  estimation  errors  involved  with  vehicle- 
referencing  increase  dramatically  as  the  actual  side  slope  angle  increases.  The  angle  estimation 
errors  are  nearly  as  large  as  the  actual  side  slope  angles. 


ROLL  AXIS 


Figure  4-5.  Roll  angle  cell  mean  estimation  errors  versus  actual  angles. 


* 


A  closer  examination  of  the  critical  side  slope  region  is  made.  For  this  study,  the  critical 
region  for  side  slope  is  defined  as  greater  than  10  degrees.  The  mean  roll  estimation  errors  for 
gravity-  and  vehicle-referencing  methods  for  the  critical  region  are  shown  in  table  4-7.  The 
average  vehicle-referencing  estimation  error  is  12.4  degrees  and  the  average  angle  in  the  critical 
region  is  12.9  degrees.  The  average  gravity-referencing  estimation  error  is  3.9  degrees  and  the 
average  angle  in  the  critical  region  is  13.2  degrees. 


16 


Table  4-7.  Reference  method  means  for  critical  region  roll  estimation  error. 


Reference 

Method 

Count 

Mean 

Std.  Dev. 

Std.  Error 

Vehicle 

156 

12.445 

4.428 

0.355 

Gravity 

240. 

3.859 

2.996 

0.193 

Dependent:  Roll ! 

Estimation  Error  in  the  Critical  Region  (degrees) 

The  critical  region  mean  estimation  error  associated  with  vehicle-referencing  is  nearly  as 
large  as  the  corresponding  mean  actual  angle.  The  critical  region  mean  error  associated  with 
gravity-referencing  is  consistent  with  the  mean  estimation  error  for  the  entire  range  of  data  and 
does  not  correspond  to  the  actual  angle  mean  for  the  critical  region. 

4.6.2.2  Pitch  Axis.  The  ANOVA  for  the  pitch  axis  data  is  shown  in  table  4-3.  The  high  F 
value  and  the  p  value  suggest  that  the  null  hypothesis,  stated  in  Section  4.5,  Experimental  Analy¬ 
sis,  can  be  rejected  with  a  confidence  level  of  99.99  percent.  Thus,  there  is  statistically  signifi¬ 
cant  evidence  that  estimation  error  variance  is,  at  least  in  part,  due  to  different  referencing 
methods. 

The  mean  vehicle  pitch  estimation  errors  are  illustrated  in  table  4-8  for  gravity-  and  vehicle- 
referenced  sensors. 


Table  4-8.  Reference  method  means  for  pitch  estimation  error. 


Reference 

Method 

Count 

Mean 

Std.  Dev. 

Std.  Error 

Vehicle 

504 

7.112 

4.747 

0.211 

Gravity 

504 

5.698 

4.031 

0.180 

Dependent:  Pitch  Estimation  Error  (degrees) 


The  gravity-referencing  technique  is  supplying  more  orientation  cues  than  the  vehicle- 
referencing  method.  However,  the  difference  in  mean  pitch  estimation  errors  is  not  as  dramatic 
as  the  difference  in  mean  roll  estimation  errors. 

Figure  4-6  shows  pitch  cell  mean  (average  estimation  error  calculated  for  each  actual  angle) 
estimation  errors  versus  the  actual  angle.  Errors  associated  with  both  gravity-  and  vehicle- 
referencing  display  the  same  upward  slope,  which  is  especially  evident  in  the  positive  angle 
range.  The  positive  slope  is  attributed  to  the  orientation  estimation  device,  the  model  vehicle 
shown  in  figure  4-2.  The  model  vehicle  was  suspended  in  front  of  the  test  subject  (see  fig¬ 
ure  4-3).  The  two-axis  inclinometer  was  mounted  just  aft  of  the  center  line,  causing  the  back  of 
the  vehicle  to  be  heavier  than  the  front.  Thus,  it  required  less  effort  for  test  subjects  to  rotate  the 
vehicle  toward  themselves,  indicating  an  up  slope,  than  to  rotate  the  vehicle  away  from  them¬ 
selves,  indicating  a  down  slope.  Since  the  effect  is  the  same  for  both  referencing  methods,  it  does 
not  alter  the  conclusions  drawn.  It  does,  however,  enhance  the  underestimation  errors  made  in 
the  positive  angle  (up  slope)  region  and  diminish  the  underestimation  errors  made  in  the  negative 
angle  (down  slope)  region.  Taking  this  into  account,  the  cell  mean  errors  associated  with  the 
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gravity-referencing  method  are  fairly  consistent  and  have  only  a  minor  dependence  on  the  actual 
angles.  The  cell  mean  errors  associated  with  the  vehicle-referencing  method  increase  proportion¬ 
ally  as  the  actual  angles  increase.  The  mean  errors  at  small  actual  angles  drop  below  the  cell 
mean  errors  associated  with  gravity-referencing. 


PITCH  AXIS 


Figure  4-6.  Pitch  angle  cell  mean  estimation  errors  versus  actual  angles. 

Finally,  the  critical  region  performance  is  explored.  The  critical  region  for  pitch  analysis  is 
defined  as  greater  than  10  degrees  up  or  down  slope.  The  mean  pitch  estimation  for  the  critical 
region  is  shown  in  table  4-9  for  both  referencing  methods.  The  average  vehicle-referencing 
estimation  error  is  8.9  degrees  and  the  average  angle  in  the  critical  region  is  12.9  degrees.  The 
average  gravity -referencing  estimation  error  is  6.1  degrees  and  the  average  angle  in  the  critical 
region  is  11.7  degrees. 


Table  4-9.  Reference  method  means  for  critical  region  pitch  estimation  error. 


Reference 

Method 

Count 

Mean 

Std.  Dev. 

Std.  Error 

Vehicle 

156 

8.945 

5.070 

0.406 

Gravity 

144 

6.111 

4.104 

0.342 

Dependent:  Pitch  Estimation  Error  in  the  Critical  Region  (degrees) 
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The  mean  estimation  errors  for  both  gravity-  and  vehicle-referencing  vary  minimally  from 
the  mean  errors  involved  with  the  entire  angle  data  range  (i.e.,  0  to  20  degrees  up  and  down 
slope).  This  finding  coupled  with  the  cell  mean  plot  information  indicates  trends  in  the  estima¬ 
tion  error  data.  The  angle  estimation  errors  associated  with  the  gravity-referencing  method  are 
consistent  and  nearly  independent  of  the  actual  angles.  The  angle  estimation  errors  associated 
with  the  vehicle-referencing  method  are  directly  proportional  to  the  actual  angles.  The  estimation 
errors  are  less  than  those  associated  with  gravity-referencing  for  small  actual  angles  and  greater 
for  large  actual  angles. 

4.6.3  Test  Subject  Effect 

The  variance  in  performance  between  test  subjects  is  shown  by  tables  4-2  and  4-3  to  be  a 
statistically  significant  source  of  total  variance.  The  goal  of  this  effort  is  not  to  investigate 
operator  characteristics  that  are  a  factor  in  vehicle  attitude  awareness.  The  analysis  of  test  subject 
effect  does  not  affect  the  conclusions  of  this  report  and,  although  it  offers  an  interesting  area  of 
study,  it  will  not  be  pursued  in  this  effort. 

4.6.4  Test  Subject  Confidence  Level 

On  the  posttest  questionnaire,  test  operators  were  asked  to  declare  the  level  of  confidence 
that  they  had  in  the  estimates  they  had  just  made.  The  scale  ranged  from  1  (low)  to  5  (high).  The 
underlying  assumption  to  the  question  is  that  the  more  confident  operators  are  in  their  under¬ 
standing  of  the  remote  vehicle  attitude,  the  more  likely  they  are  to  take  action  based  on  that 
understanding.  If  operators’  estimations  are  good  and  they  are  confident  in  them,  they  will  be 
able  to  plan  successful  courses  of  action.  If  they  are  not  confident,  they  will  be  hesitant  to  take 
action  even  when  their  estimates  are  good. 

Confidence  levels  for  the  gravity-referencing  method  averaged  3.9  compared  with  2.5  for  the 
vehicle-referencing  method.  Eleven  test  subjects  reported  a  higher  confidence  rating  for 
gravity-referencing  than  for  vehicle-referencing.  One  test  subject  gave  the  two  methods  equal 
ratings.  Results  are  presented  in  figure  4-7. 

4.6.5  Test  Subject  Workload  Rating 

Remote  vehicle  operators  have  a  heavy  workload.  The  goal  of  this  effort  is  to  improve 
operators’  understanding  of  vehicle  attitude  with  minimal  impact  on  their  workload.  When  test 
operators  finished  each  session,  they  were  asked  to  rate  the  workload  of  the  task  on  a  scale  of  1 
(easy)  to  5  (difficult).  The  results  are  charted  in  figure  4-8. 

The  workload  ratings  averaged  3.5  for  the  gravity-referencing  method  and  3.0  for  the 
vehicle-referencing  method.  These  results  provide  evidence  that  the  gravity-referencing  method 
slightly  increases  operator  workload.  The  increase,  however,  is  minimal. 
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WORKLOAD  RATING  CONFIDENCE  LEVEL 


TEST  OPERATOR 


GRAVITY-REFERENCE 

VEHICLE-REFERENCE 


5  i 


5  6  7  8  9 

TEST  OPERATOR 


10  11  12 


AVG 


GRAVITY-REFERENCE 

VEHICLE-REFERENCE 


Figure  4-8.  Workload  rating  for  gravity-  and  vehicle- 
referencing  methods. 
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5.  RECOMMENDATIONS  AND  CONCLUSIONS 


5.1  IMPLEMENTATION  SUGGESTIONS 

The  hood  of  the  remote  vehicle  appeared  in  the  visual  work  space  on  the  lower  third  of  the 
video  screen.  It  created  a  two-dimensional  artificial  horizon,  supplying  the  operator  with  remote 
vehicle  attitude  information.  The  results  determined  in  this  investigation  concerning  the  effec¬ 
tiveness  of  gravity-referencing  the  sensor  package  are  not  applicable  to  a  system  in  which  the 
hood  or  other  significant  part  of  the  vehicle  front  is  not  visible  in  the  video  image. 

5.1.1  Graphical  Overlay 

As  described  in  the  previous  section,  the  hood  of  the  remote  vehicle  served  as  an  artificial 
horizon.  The  roll  axis  of  the  horizon  indicator  was  a  one-to-one  angular  position  mapping  of  the 
roll  of  the  vehicle.  As  a  result,  the  roll  angle  estimation  errors  were  minimal.  The  pitch  axis  of 
the  hood-generated  indicator  was  an  angular-to-linear  mapping  of  the  pitch  of  the  vehicle.  As  the 
vehicle  encountered  an  up  slope,  the  hood  filled  more  than  one-third  of  the  video  image.  When 
the  vehicle  went  down  hill,  the  hood  filled  less  than  one-third  of  the  image.  A  simple  graphic 
indicator  could  be  overlaid  on  the  edge  of  the  video  display  to  enhance  angular  position  informa¬ 
tion  provided  by  the  linear  movement  of  the  hood  in  the  video  image. 

5.1.2  Three  Degree-of-Freedom  Platform 

The  gravity-referencing  method  can  be  implemented  as  a  two-degree-of-freedom  stand-alone 
platform,  such  as  the  prototype  mechanism.  This  design  is  especially  advantageous  for  retrofit¬ 
ting  teleoperator  systems. 

Most  remote  vehicles  are  equipped  with  a  yaw-pitch  (commonly  called  pan  and  tilt) 
mechanism.  A  pitch-roll  referencing  platform  positioned  under  a  yaw-pitch  platform  results  in  a 
redundant  degree-of-freedom  in  the  pitch  axis.  The  two  mechanisms  can  be  replaced  by  one 
three-degree-of-freedom  platform,  affording  a  reduction  in  cost  and  complexity.  The  operator- 
commanded  “head”  motions  would  be  combined  with  the  gravity-referencing  motions  to 
command  the  movement  of  the  platform.  To  achieve  a  level  pan  arc,  the  three-degree-of-freedom 
platform  should  be  ordered  with  the  roll  axis  adjacent  to  the  vehicle  base,  the  pitch  axis  next,  and 
the  yaw  axis  on  top. 

5.2  SUMMARY 

The  operator  of  a  teleoperated  land  vehicle  must  have  a  sufficient  understanding  of  the 
remote  vehicle  attitude  to  successfully  operate  it  in  an  unstructured  outdoor  environment.  The 
goal  of  this  effort  was  to  develop  a  human-machine  interface  that  provides  sufficient  remote 
vehicle  attitude  information  while  minimally  impacting  operator  workload. 

The  method  of  gravity -referencing  the  remote  sensor  package  was  implemented.  The 
effectiveness  of  this  method  versus  vehicle-referencing  the  sensors  was  experimentally  investi¬ 
gated. 

Statistically  significant  improvement  (with  a  99.9  percent  confidence  level)  was  observed  in 
the  operator’s  understanding  of  remote  vehicle  attitude  when  the  sensor  package  was  gravity- 
referenced  compared  with  when  it  was  vehicle-referenced. 
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Experimental  results  showed  a  significantly  higher  level  of  operator  confidence  in  vehicle 
attitude  awareness  with  the  gravity-referencing  method.  A  minimal  increase  in  operator  work¬ 
load  was  also  documented  for  the  method  of  gravity-referencing. 

Gravity-referencing  the  sensor  package  presents  an  added  cost  in  developing  or  retrofitting  a 
remote  vehicle.  Statistically  significant  improvement  is  not  a  sufficient  reason  to  implement  the 
gravity-referencing  method.  The  new  method  must  provide  an  improvement  in  vehicle  attitude 
awareness  of  “operational  significance”  to  warrant  its  implementation.  Unlike  statistical 
significance,  operational  significance  is  not  determined  by  a  defined  method  or  formula;  it  is  a 
judgment  based  on  the  consideration  of  the  data  from  an  operational  perspective.  Vehicle  attitude 
awareness  is  most  important  when  the  terrain  slope  magnitudes  pose  a  hazard  to  the  effective 
operation  of  the  remote  vehicle.  The  effectiveness  of  the  gravity-referencing  method  was 
compared  with  that  of  the  vehicle-referencing  method  for  slopes  exceeding  10  degrees  (referred 
to  as  the  critical  region). 

From  the  information  presented  in  table  4-7  and  accompanying  text,  the  mean  roll  angle 
estimation  error  for  the  gravity-referencing  method  is  30  percent  of  the  actual  angle.  The  mean 
roll  angle  estimation  error  for  the  vehicle-referencing  method  is  96  percent  of  the  actual  angle. 
Consider,  for  example,  that  if  an  actual  side  slope  of  20  degrees  is  encountered  by  a  remote 
vehicle  with  vehicle-referenced  sensors,  an  operator  would  estimate  the  slope  as  only  0.8  degree. 
If  the  same  terrain  were  encountered  by  a  remote  vehicle  with  gravity-referenced  sensors,  an 
operator  would  estimate  the  slope  as  14  degrees.  With  sensor  gravity-referencing  implemented, 
the  improvement  in  the  operator’s  understanding  of  the  vehicle  roll  angle  is  operationally 
significant. 

The  same  comparison  is  considered  for  the  pitch  axis  of  the  remote  vehicle.  The  mean  pitch 
angle  estimation  error  for  the  gravity-referencing  method  is  52  percent  of  the  actual  angle.  The 
mean  pitch  angle  estimation  error  for  the  vehicle-referencing  method  is  69  percent  of  the  actual 
angle.  These  percentages  are  based  on  the  information  in  table  4-10  and  related  analysis.  Again, 
consider  that  if  a  remote  vehicle  with  vehicle-referenced  sensors  encountered  an  actual  side 
slope  of  20  degrees,  an  operator  would  estimate  the  slope  as  only  5.2  degrees.  If  the  same  terrain 
were  encountered  by  a  remote  vehicle  with  gravity-referenced  sensors,  an  operator  would 
estimate  the  slope  as  9.6  degrees.  The  gravity-referencing  method  provides  an  improvement  in 
pitch  angle  estimation,  but  the  operational  significance  is  not  obvious.  One  idea  for  enhancing 
the  effectiveness  of  the  gravity-referencing  method  for  pitch  angle  estimation  is  presented  and 
discussed  in  Section  5.1.1,  Graphical  Overlay.  Although  the  graphical  indicator  has  not  been 
formally  tested,  informal  investigation  indicates  that  the  gravity-referencing  method  coupled 
with  the  simple  graphical  indicator  will  provide  an  operationally  significant  improvement  in 
pitch  angle  estimation. 
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Appendix 

EXPERIMENT  DOCUMENTS 


UNMANNED  GROUND  VEHICLE  HUMAN-MACHINE  INTERFACE 

ATTITUDE  AWARENESS  STUDY 


Explanation  of  Test  Procedures 

The  experiment  described  in  this  document  is  entitled  “UGV  Human-Machine  Interface 
Attitude  Awareness  Study.”  The  experiment  is  designed  to  study  the  effectiveness  of  two 
concepts  for  presenting  remote  vehicle  attitude  (pitch  and  roll)  information  to  the  operator.  (A 
remote  vehicle  is  an  unmanned  mobile  platform  that  is  being  controlled  from  some  distance 
away  by  an  operator.  An  operator  uses  visual  and  binaural  displays  of  sensors  mounted  on  the 
remote  vehicle  to  make  operation  decisions.) 

For  the  experiment,  you  will  be  presented  with  visual  and  binaural  (audio)  displays  of  sensor 
data  collected  from  an  HMMWV  vehicle  using  two  methods.  The  HMMWV  was  driven  off-road 
over  rough  terrain  and  video,  audio,  pitch,  and  roll  data  were  recorded.  You  will  not  be  directly 
operating  or  teleoperating  (remote  controlling)  the  HMMWV  vehicle.  The  video  you  will  be 
viewing  and  audio  you  will  be  hearing  were  prerecorded.  There  are  no  vehicle  safety  issues  to  be 
concerned  with  during  the  experiment. 

You  will  participate  in  two  test  sessions,  each  session  will  feature  a  different  method  of 
presenting  remote  vehicle  attitude  information  via  the  video  and  audio  displays.  Each  test  session 
will  consist  of  14  prerecorded  driving  scenarios,  each  45  seconds  in  length.  At  three  random 
points  during  each  45-second  scenario,  the  video  and  audio  will  be  stopped,  and  you  will  make 
an  estimate  of  the  orientation  with  respect  to  gravity,  (pitch  and  roll  angles)  of  the  HMMWV 
vehicle.  You  will  express  your  estimate  by  positioning  a  model  vehicle  in  the  same  orientation  as 
the  observed  HMMWV  vehicle,  then  pressing  a  button. 

There  will  be  a  brief  practice  session  during  which  you  will  have  the  opportunity  to  become 
familiar  with  the  test  equipment  and  procedures.  The  practice  session  will  take  approximately  ten 
minutes.  The  test  sessions  will  require  between  20  and  25  minutes  each. 

Your  participation  is  important  and  greatly  appreciated!  The  information  derived  from  this 
experiment  will  assist  engineers  and  scientists  in  designing  and  building  better  remotely  operated 
vehicles. 

♦  Please  perform  as  well  as  you  can.  Remember,  we  are  not  analyzing  your  performance  to 

compare  with  that  of  other  test  subjects;  we  are  interested  only  in  comparing  design  features  of 
the  hardware  used  to  collect  and  display  vehicle  attitude  information. 


25 


UNMANNED  GROUND  VEHICLE  HUMAN-MACHINE  INTERFACE  ATTITUDE 

AWARENESS  STUDY 


Pretest  Questionnaire 
Name: 

Phone  (work):  Age:  Sex: 

Occupation: 

1.  Are  you  licensed  to  drive  a  motor  vehicle? 

If  yes,  how  many  years? 

2.  Are  you  familiar  with  an  HMMWV  as  a 

driver? 
passenger? 
direct  observer? 
other  (please  list)? 

3.  Do  you  have  problems  with  your  vision  (that  have  not  been  corrected  with  aids)? 
If  yes,  please  explain: 

4.  Do  you  have  problems  with  your  hearing  (that  have  not  been  corrected  with 
aids)? 

If  yes,  please  explain: 

5.  How  susceptible  to  motion  sickness  are  you?  (circle  one) 

Extremely 

Very 

Moderately 
Minimally 
Not  at  all 

6.  What  is  your  off-road  driving  experience?  (circle  one) 
none 

minimal:  driver  passenger  both 

moderate:  driver  passenge  both 

substantial:  driver  passenger  both 
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UNMANNED  GROUND  VEHICLE  HUMAN-MACHINE  INTERFACE  ATTITUDE 

AWARENESS  STUDY 


» 


Posttest  Questionnaire 

1.  Rate  your  confidence  level  for  the  vehicle  orientation  estimates  you  just  made. 


2. 


LOW 

(guess) 

12  3  4 

Did  you  like  this  view  from  the  vehicle? 
NOT-AT-ALL 

12  3  4 


HIGH 

(very  confident) 
5 


VERY  MUCH 
5 


3.  Did  you  experience  any  physical  ill  effects  (eye  strain,  nausea,  headaches,  etc.)? 
If  so,  please  list  them  and  rate  the  severity.  Did  any  of  these  symptoms  impair 
your  ability  to  do  the  task? 


SYMPTOM  ILL  EFFECT 

IMPAIRED  ABILITY 

noticeable  severe 

no  slightly  yes 

1  2  3  | 

A  B  C 

12  3  1 

A  B  C 

1  2  3  | 

A  B  C 

4. 

Rate  the  task  workload  (level  of  calculation,  determination,  cognitive  effort)  of 
making  the  orientation  estimates. 

LIGHT 

HEAVY 

(very  easy) 

(very  hard) 

12  3  4 

5 

5.  Please  list  your  comments  and  observation: 


# 


A 
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